In a known image pickup apparatus, a single image pickup element is used, and the pupil of the pickup lens is divided to capture a pair of images having parallax for displaying a stereoscopic image (for example, see JP 2011-515045 A and JP 2003-523646 A). The image pickup element used in this image pickup apparatus is provided with a plurality of microlenses in a two-dimensional array and a pair of light-receiving elements corresponding to each microlens and is disposed so that the pupil divided image of the pickup lens forms on each pair of light-receiving elements. In this way, a pixel signal to generate one captured image of a stereoscopic image is obtained from one light-receiving element, and a pixel signal to generate the other captured image of a stereoscopic image is obtained from the other light-receiving element.